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(54) Adjusting color in digital images 

(57) Color data of pixels of a digital image are ad- 
justed by identifying pixels of a digital image having orig- 
inal color data corresponding to predetermined color 



and shape characteristics and adjusting the original 
color data of the identified pixels to achieve a desired 
result. 
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Description 

Background 

5 The invention relates to adjusting color in digital images. 

A digital image is a raster of rows and columns of picture elements, or "pixels", each of which include information 
such as color data. Color data describes the pixel color using any of a variety of color systems. For example, in the 
RGB (red-green-blue) system, colors are represented as a combination of red, green, and blue components. Color 
data for a pixel thus includes numerical values reflecting the intensities of the red, green, and blue components of the 

10 pixel color. Other color systems include CMYK (cyan-magenta-yellow-key) and HSV (hue-saturation-value), which 
similarly represent colors as combinations of their respective color components. 

Applications exist that allow a user to adjust the color of a digital mage. In some applications, the user can manually 
adjust the color of a pixel by methods such as replacing the existing color data with the desired color data, enhancing 
or reducing a specified color component, or mixing the existing color data with color data for another color. However, 

*5 it can be a time consuming process for the user to identify specific pixels and to adjust the color data of those pixels 
until the desired color is achieved. 

Summary 

20 in general, in one aspect, the invention features a computer-implemented method to adjust color data of pixels of 

a digital image by identifying pixels of a digital image having original color data corresponding to predetermined color 
and shape characteristics and adjusting the original color data of the identified pixels to achieve a desired result. 

Certain implementations of the invention may include one or more of the following features. The method includes 
defining a portion of a digital image in response to user input, where the identification of pixels identifies only pixels of 
25 the digital image within the user-defined, portion. 

Identifying pixels of a digital image having original color data corresponding to predetermined color and shape 
characteristics filters the digital image based on predetermined color characteristics and using filters corresponding to 
predetermined shape characteristics. Pixels of the digital image having original color data corresponding to predeter- 
mined color and shape characteristics are identified by creating color enhanced data for the pixels of the digital image 
so based on the original color data, and filtering the color enhanced data for the digital image based on predetermined 
color characteristics and using filters corresponding to predetermined shape characteristics. The digital image may be 
filtered by convolving color data of the pixels of the digital image with the filters. 

The method includes verifying that the identified pixels actually correspond to predetermined color and shape 
characteristics, based on user input. 
35 The predetermined color and shape characteristics correspond to a red eye effect. The identified pixels correspond 

to a portion of an eye in the digital image, and adjusting the original color data of the identified pixels determines an 
iris color, reduces a red component in the original color data of the identified pixels, and modifies the original color data 
towards achieving the iris color. Determining the iris color includes identifying pixels of the digital image corresponding 
to an iris area of an eye affected by the red eye effect, and evaluating the color data of the iris pixels to determine the 
40 iris color. Determining the iris color may alternatively use a default color, which may be selected by a user. 

In general, in another aspect, the invention enables use of the computer-implemented method using a memory 
device storing computer-readable instructions for aiding a computer to adjust color data of pixels of a digital image. 

In general, in another aspect, the invention provides an apparatus to adjust color data of pixels of a digital image, 
comprising a storage medium to store a digital image and a processor operatively coupled to the storage medium and 
45 configured to perform the computer-implemented method. 

Among the advantages of the invention are one or more of the following. The invention identifies pixels of color 
images to be adjusted for the user. The invention automatically adjusts the color data of identified pixels. 

Other features and advantages of the invention will become apparent from the following description and from the 
claims. 

50 

Brief Description of the Drawings 

Figure 1 is an illustration of an eye. 
Figure 2 is an illustration of the red eye effect on an eye. 
55 Figure 3 illustrates a computer system suitable for use with the present invention. 

Figure 4 is a flow diagram of a method for adjusting color to minimize the red eye effect in a digital image in 
accordance with the present invention. 

Figure 5 is a flow diagram of a method for identifying red eye area candidates in a digital image in accordance 
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with the present invention. 

Figures 6-8 illustrate filters used in accordance with the present invention to identify red eye area candidates in a 
digital image. 

Figure 9 is a flow diagram of a method for adjusting color in identified red eye areas in a digital image in accordance 
5 with the present invention. 

Description 

Automatic color adjustment in digital images is desirable in some situations. For example, automatic color adjust- 

10 ment may be desired to adjust recurring and identifiable photographic discolorations. An example of such a discoloration 
is the "red eye" effect that often occurs in photographs. Figure 1 illustrates the natural coloring of an eye 100, having 
a colored iris 110 and a blackish pupil 120. Figure 2 illustrates an eye 200 with the red eye effect, in which a red circular 
shape 220 at least partially (and sometimes completely) obscures the iris 210 and typically obscures the entire pupil 
120 (Figure 1). A small white circular glare 230 often appears in the center of the red eye portion 220. 

15 Occurrences of the red eye effect share these common features and thus are identifiable by a red circular area 

220, frequently having a white center 230 and surrounded by a non-red background 210. In some cases, the non-red 
background 210 may be a ring of the iris color as shown in Figure 2; in other cases, if the red circular area 220 obscures 
the entire iris, the non-red background 210 may be the whites of the eye. The invention provides a method for auto- 
matically detecting and adjusting the color of such identifiable effects. 

20 The invention may be implemented in special purpose hardware, software applications running on general purpose 

hardware, or a combination of both. Preferably, the invention is implemented in a software application executing in a 
computer system. For example, the invention may be implemented as a feature in image processing applications such 
as the PhotoDeluxe® computer program, available from Adobe Systems Incorporated of San Jose, California. 

Figure 3 illustrates an appropriate computer system 300, including a CPU 310, a RAM 320, and an I/O controller 

25 330 coupled by a CPU bus 340. The I/O controller 330 is also coupled by an I/O bus 350 to input devices such as a 
keyboard 360 and a mouse 370, and output devices such as a monitor 380. 

Referring to Figure 4, a user selects an area in a digital image for color adjustment (step 410). The area selected 
may enclose just a single eye, two or more eyes, or the entire digital image. Area selection may be implemented using 
various methods, such as by providing a user interface that allows the user to define the vertices or edges of a rectangle. 

30 Candidate areas within the user-selected area of the image are then automatically identified (step 420). various 

identification methods may be used. Referring to Figure 5, one method to identify the red eye effect is to color enhance 
the user-selected area of the digital image to more clearly distinguish pixels having specific color characteristics (step 
421). To identify the red eye effect, the color red may be enhanced in the image to more easily identify red circular 
areas 220. 

35 Various color enhancement methods may be used. If the digital image is defined using the RGB color system, the 

color data for each pixel includes values r, g, and 6, respectively representing red, green, and blue components for 
the pixel. A method of red enhancement may assign a value to each pixel in the selected area based on the relative 
intensity of its red component. 

For example, a pixel having a red color will have a greater value for its red component rthan for either of its green 

40 g or blue b components, and thus may be identified if its color data satisfies the relations r>k^g and r> k 2 b, where ^ 
and k 2 are predetermined constants. If a pixel does not satisfy these relations, the pixel color is determined not to be 
red, and a red-enhanced value c r of 0 is assigned to the pixel. 

On the other hand, a pixel satisfying the equations is determined to be red, and a red-enhanced value c r may be 
calculated as follows: 

45 

c - (r-g) + (r-b) 

r 3* (r + g+ b+ avg. intensity) 

where avg. intensity represents the average intensity of the pixels of the user-selected area, calculated as the average 
so sum of the r, g, and b components of each pixel of the area. This formula may be modified as desired. 

Based on the red-enhanced values c r candidate areas that may exhibit the red eye effect are identified. Again, 
various methods may be used. One method calculates convolutions of the red-enhanced image data with a filter (step 
422 of Figure 5) such as that illustrated in Figure 6. The filter 600 consists of data bits corresponding to pixels of the 
user-selected image area, where data bits of the filter area 61 0 are assigned a value of 1 and data bits in the surrounding 
55 area 620 are assigned a value of 0. 

In general, the size, location, and number of areas in the image affected by the red eye effect are unknown. For 
this reason, filters of varying sizes are tested within the user selected area of the digital image. Using a filter such as 
that illustrated in Figure 6, convolutions are calculated as dot products between the various filters and the red-enhanced 
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image data. The dot product indicates the correspondence between the filter and image pixels identified as having a 
red color (non-zero red-enhanced image data). 

Referring again to Figure 5, candidates for red eye areas are identified based on the convolution calculations (step 
423) . various tests may be used. For example, a test may determine that a red eye area candidate is identified when 
5 the convolution calculation exceeds a predetermined threshold. Another test may determine that a red eye candidate 
is identified when the convolution calculation is maximized across the various filter sizes and positions. 

Alternatively, steps 421-423 of Figure 4 may be modified to identify not only red circular areas (220 in Figure 2), 
but red circular areas 220 with non-red backgrounds 210. 

For example, pixels in areas corresponding to the non-red background 210 typically have a weak color, such as 
10 black, white, a shade of grey, brown or a light blue or green. Represented in RGB, weak colors have similar values for 
each of the red, green, and blue color components, and pixels having weak colors may be distinguished from pixels 
having strong colors by deriving another set of color enhanced data (step 421 of Figure 4) for the image. For example, 
pixels may be assigned a color enhanced value c w as follows: 

15 

c - = I (r - a) t (r - b) ± (a - b) I 

3 * (r + gr + jb + avg. intensity) 

20 where avg. intensity is calculated as explained above. Thus, c w has relatively large values for pixels of strong color, 

and relatively small values for pixels of weak color. 

The dot product of the weak-color enhanced color data and a ring shaped filter 700 such as that illustrated in Figure 

7, where data bits of areas corresponding to the ring 720 have a value of -1 and in other areas 710, 730 have a value 

of 0, produces a value with a small negative value when the filter area 720 corresponds to areas in the image having 
25 a weak color, and a larger negative value when the filter area 720 corresponds to areas in the image having stronger 

color. As with the filter 600 of Figure 6, dot products are calculated using filters 700 of different sizes and positions. 
To identify areas in the image having a red circular area 220 surrounded by a weakly colored background 210, a 

convolution can be calculated as the sum of two dot products: (1) the dot product of the red enhanced color data for 

the image with the circular filter of Figure 6; and (2) the dot product of the weak-color enhanced color data for the image 
30 with the ring shaped filter of Figure 7. Recalling that a red eye portion includes a red eye ring 220 surrounded by a 

weak colored area 210, the convolution is calculated with filters having corresponding sizes and positions. Specifically, 

as shown in Figure 8, for a circular filter area 610 having a radius of r, ring shaped filter area 720 has an inner radius 

of r and is positioned with the circular filter area 610 at its center. 

Again, various tests may be used to determine whether the convolution identifies a possible red eye area candidate. 
35 For example, candidates may be determined based on whether their convolution calculation is maximized or exceeds 

a predetermined threshold. 

Returning again to Figure 4, once a red eye area candidate is identified based on the convolution data (step 420), 

an additional interactive test may be optionally provided to allow the user to verify whether the identified red eye area' 

candidate actually corresponds to a red eye area (step 430). For example, when a convolution calculation for specific 
40 filters exceeds a predetermined threshold, a user interface may be implemented to highlight the area of the digital 

image corresponding to the area identified by the circular filter of Figure 6, and may display a dialog box asking the 

user to confirm whether the indicated area exhibits the red eye effect. 

Once a red eye area is identified (and optionally verified) (steps 420 and 430), the color data for pixels within that 

area are evaluated and adjusted to reduce the red eye effect (step 440). 
45 The specific method by which the pixel data is altered may be varied. For example, referring to Figure 9, the natural 

color of the iris may be determined (step 441) and the pixels affected by the red eye effect modified to reduce the 

redness by altering their color data based on the determined iris color (step 442). 

As shown in Figure 2, in some cases, the iris 210 surrounds the red eye circular area 220. Thus, original color 

data of the image (not the color enhanced data of the image) of pixels within an area immediately surrounding the red 
50 eye area identified in steps 420 and 430 of Figure 4 may be evaluated to determine the average ratio of r.g and r. 6 for 

the iris color (step 441 of Figure 9). Original color data of each pixel in the red eye area is next identified, and if a pixel's 

color is identified as red (having a strong red component r), new values /\ p/, and b' are calculated (step 442) to 

approximate the ratios calculated for the iris color (step 441). 

One method for adjusting the color determines new values /*, c/, and tt that reduce the overall intensity (which is 
55 typically disproportionately large in the red eye area) and approximates the ratios of the color components of the iris 

color by solving the following set of equations: 
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r + g + b = k (r' + g' + Jb') 
r : g = r' : g' 

s r : b = r' : b' 

where k is a constant less than one. 

In some cases, however, the red eye circular area 220 may obscure most or all of the Iris area 210, such that there 
are not enough iris colored pixels to properly identify the iris color. In such a case, the iris color used to replace the 

io color data for red pixels in the identified red eye area (step 442 of Figure 9) may be a default color. For example, 
because the red eye effect occurs most frequently with light colored eyes, a light blue or grey color may be used. 
Alternatively, a user interface may be provided to allow a user to set a default color. As another alternative, the user 
interface may be provided to identify instances where the iris color is not identifiable and to allow the user to specify 
an iris color for the specific instance. The user may specify a color by, for example, selecting or mixing colors from a 

'5 displayed palette or selecting the color of a displayed pixel. 

As discussed above and as illustrated in Figure 2, the red eye effect typically obscures the pupil area 100 (Figure 
1). Thus, the pupil area is recreated (step 443 of Figure 9) by assigning color data for a blackish color to pixels in the 
identified red eye area. The specific size and location of the recreated pupil area may vary. For example, the pupil may 
be created as a circular area centered in the recreated iris area, having a diameter that is one-third of the diameter of 

20 the recreated iris. A more complex approach calculates the overall average brightness of pixels in the image, and 
creates the pupil such that its size reflects the average amount of light detected in the image scene. 

An additional feature common to the red eye effect is the white circular glare 230, shown in Figure 2. Completely 
eliminating the glare can give an artificial appearance to the image, but it may be aesthetically desirable to reduce the 
intensity of the glare area (step 444 of Figure 9). In the RGB system, pixels in the glare area are identifiable by positions 

25 within the identified red eye area having high r, p, and b values. The color data for such pixels may be modified using 
various methods, such as by reducing the intensity of the r, g, and b values, or by replacing the color data with prede- 
termined values / J , gf, and U for a white color of moderate intensity. 

Referring again to Figure 4, when the red eye adjustment is complete (step 440), steps 420, 430 and 440 may be 
repeated until the entire selected area (step 410) is evaluated using the range of filter sizes and positions. If desired, 

30 termination conditions may be provided. For example, the invention may be implemented to identify only a predeter- 
mined number of areas for color adjustment, where the predetermined number may be default number, or a number 
specified by the user when selecting the area in step 410. 

Other embodiments are within the scope of the following claims. The invention may be implemented to adjust color 
data for other recognizable features. For example, photocopies of three-hole-punched paper have dark circles in the 

■35 locations of the holes. The invention may be implemented to identify pixels comprising such circles and to adjust the 
color data for those pixels. 

Although the invention has been described for color data represented in the RGB color system, the invention may 
be implemented for use with other color systems as well. 

Various filters and convolution functions may be provided as needed. For example, the filters of Figures 6 and 7 
40 may be used in combination or alone to identify the red eye portion. Additionally, additional color enhancement may 
be performed and the convolution may consider an extra filter to identify the white circular glare in the center of a red 
eye ring. For identifiable effects other than the red eye effect, filters and convolution functions may be provided as 
needed. 

45 

Claims 

1. A computer-implemented method to adjust color data of pixels of a digital image, comprising: 

50 identifying pixels of a digital image having original color data corresponding to predetermined color and shape 

characteristics; and 

adjusting the original color data of the identified pixels to achieve a desired result. 

2. The method of claim 1 , further comprising defining a portion of a digital image in response to user input, wherein 
55 identifying pixels of a digital image having original color data corresponding to predetermined color and shape 

characteristics identifies only pixels of the digital image within the user-defined portion. 

3. The method of claim 1 , wherein identifying pixels of a digital image having original color data corresponding to 
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predetermined color and shape characteristics further comprises filtering the digital image based on predetermined 
color characteristics and using filters corresponding to predetermined shape characteristics. 

4. The method of claim 3, wherein 

5 

identifying pixels of a digital image having original color data corresponding to predetermined color and shape 
characteristics further comprises creating color enhanced data for the pixels of the digital image based on the 
original color data, and 

filtering the digital image based on predetermined color characteristics and using filters corresponding to pre- 
10 determined shape characteristics filters the color enhanced data for the digital image. 

5. The method of claim 3, wherein filtering the digital image using filters corresponding to predetermined shape char- 
acteristics further comprises convolving color data of the pixels of the digital image with the filters. 

15 6. The method of claim 1 , further comprising verifying, based on user input, that the identified pixels actually corre- 
spond to predetermined color and shape characteristics. 

7. The method of claim 1 , wherein the predetermined color and shape characteristics correspond to a red eye effect. 

20 8. The method of claim 7, wherein the identified pixels correspond to a portion of an eye in the digital image, and 
adjusting the original color data of the identified pixels further comprises: 

determining an iris color; 

reducing a red component in the original color data of the identified pixels; and 
25 modifying the original color data towards achieving the iris color. 

9. The method of claim 8, wherein determining the iris color further comprises: 

identifying pixels of the digital image corresponding to an iris area of an eye affected by the red eye effect; and 
30 evaluating the color data of the iris pixels to determine the iris color. 

10. The method of claim 8, wherein determining the iris color further comprises using a default color. 

11. The method of claim 10, wherein the default color is selected by a user. 

35 

12. A memory device storing computer-readable instructions for aiding a computer to adjust color data of pixels of a 
digital image, comprising instructions to 

identify pixels of a digital image having original color data corresponding to predetermined color and shape 
40 characteristics; and 

adjust the original color data of the identified pixels to achieve a desired result. 

13. An apparatus to adjust color data of pixels of a digital image, comprising: 

45 a storage medium to store a digital image; 

a processor operatively coupled to the storage medium and configured to: 

identify pixels of a digital image having original color data corresponding to predetermined color and shape 
characteristics; and 

50 adjust the original color data of the identified pixels to achieve a desired result. 
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